Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Report success and failures better in seqerakit #157

Merged
merged 5 commits into from
Aug 2, 2024
Merged

Report success and failures better in seqerakit #157

merged 5 commits into from
Aug 2, 2024

Conversation

ejseqera
Copy link
Member

@ejseqera ejseqera commented Jul 31, 2024 •
edited
Loading

Re #152 to improve seqerakit reporting on errors and successes.

With these changes, the command invoked and subsequent stdout will be provided through logging by default, with improved error handling if the command fails.

Running the following YAML:

launch:
  - name: "hello-world-test-seqerakit"
    pipeline: "https://github.com/nextflow-io/hello"
    workspace: "scidev/testing"
    compute-env: "seqera_aws_london_fusion_nvme"

Will return the following by default logging level of INFO upon success:

seqerakit example.yml       
INFO:root: Running command: tw launch --name hello-world-test-seqerakit --workspace scidev/testing --compute-env seqera_aws_london_fusion_nvme https://github.com/nextflow-io/hello
INFO:root: Command output: Workflow 3A8HdVzqtBDWST submitted at [scidev / testing] workspace.

    https://tower.nf/orgs/scidev/workspaces/testing/watch/3A8HdVzqtBDWST

If there is an error or the command fails:

 seqerakit example.yml
INFO:root: Running command: tw launch --name hello-world-test-seqerakit --workspace scidev/test --compute-env seqera_aws_london_fusion_nvme https://github.com/nextflow-io/hello
INFO:root: Command output: ERROR: Workspace 'test' at organization 'scidev' not found
ERROR:root:Resource creation failed: 'ERROR: Workspace 'test' at organization 'scidev' not found'. Check your config and try again.

Similarly in a Python script, without logging specified explicitly:

python test-logging.py 
Traceback (most recent call last):
  File "/Users/eshajoshi/Development/seqera-kit/dump/test-sp-class.py", line 12, in <module>
    sp.launch(
  File "/Users/eshajoshi/miniconda3/envs/dev/lib/python3.11/site-packages/seqerakit/seqeraplatform.py", line 44, in __call__
    return self.tw_instance._tw_run(command, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eshajoshi/miniconda3/envs/dev/lib/python3.11/site-packages/seqerakit/seqeraplatform.py", line 145, in _tw_run
    return self._execute_command(full_cmd, kwargs.get("to_json"), print_stout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eshajoshi/miniconda3/envs/dev/lib/python3.11/site-packages/seqerakit/seqeraplatform.py", line 114, in _execute_command
    self._handle_command_errors(str(stdout))
  File "/Users/eshajoshi/miniconda3/envs/dev/lib/python3.11/site-packages/seqerakit/seqeraplatform.py", line 134, in _handle_command_errors
    raise ResourceCreationError(
seqerakit.seqeraplatform.ResourceCreationError: Resource creation failed: 'ERROR: Workspace 'test' at organization 'scidev' not found'. Check your config and try again.

TODO:

  • Figure out how to avoid returning stdout for methods called by the Overwrite class that will invoke commands to return JSON and perform searches for entities
  • Add tests

adamrtalbot
adamrtalbot approved these changes Aug 1, 2024
View reviewed changes
Copy link
Collaborator

@adamrtalbot adamrtalbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks cleaner and better than before!

@adamrtalbot adamrtalbot mentioned this pull request Aug 1, 2024
Merged
@ejseqera ejseqera merged commit c006f67 into dev Aug 2, 2024
2 checks passed
@ejseqera ejseqera deleted the 152-report-success-failures-better branch August 2, 2024 01:20
@ejseqera ejseqera mentioned this pull request Aug 2, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@adamrtalbot adamrtalbot adamrtalbot approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ejseqera @adamrtalbot